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The  Instructional  Quality  Inventory 
(IOI)  is  a  set  of  quality  control  or 
evaluation  procedures  for  the  three 
main  parts  of  an  instructional  pro¬ 
gram;  the  objectives,  tests,  and 
instructional  materials  or  presenta¬ 
tions.  The  procedures  can  be  used 
for  quality  control  during  the  in¬ 
structional  development  process  or 
can  be  used  to  evaluate  existing  in¬ 
structional  programs.  The  IQI  proced¬ 
ures  can  also  serve  as  standards  for 
instruction  developed  under  contract. 

The  initial  work  on  the  IOI  was  done 
under  contract  to  the  U.S.  Navy  Per¬ 
sonnel  Research  and  Development  Cen¬ 
ter  (NPRDC)  by  Courseware,  Inc. 
(Merrill,  Wood,  Richards  and  Schmidt, 
1977).  The  initial  version  was  ex¬ 
tensively  revised  and  field  tested  by 
NPRDC,  and  a  final  version  was  pub- 
blished  in  1979,  (Ellis,  Wulfeck,  and 
Fredericks,  1979). 

THE  IQI  CLASSIFICATION  SCHEME 
AND  PROCEDURES 

The  IQI  was  initially  designed  to 
parallel  and  supplement  the  mili¬ 
tary’s  Instructional  Systems  Develop¬ 
ment  model  (Branson,  Rayner,  Cox, 
Furman,  King,  and  Hannum,  1975).  How¬ 


ever,  it  can  be  applied  to  any 
systematically  developed  program  of 
instruction.  The  procedures  are 
based  on  a  system  for  classifying 
objectives,  test  items,  and  instruc¬ 
tional  presentations. 

Classification  schemes  have  been 
used  in  the  past  with  some  success 
but  there  have  always  been  problems. 
In  order  for  a  classification  scheme 
to  be  useful  two  conditions  must  be 
met.  First,  instructional  and  test 
developers  must  be  able  to  make 
reliable  classifications  using  the 
scheme.  Second,  the  scheme  must  have 
clear  implications  for  instructional 
and  test  item  development;  the  impli¬ 
cations  should  be  specified  as 
prescriptions  for  development.  Typi¬ 
cally,  classification  procedures  are 
too  loosely  defined  (e.g.  ,  Gagne, 
1976;  Popham,  1977)  to  permit  reli¬ 
able  classification  by  users  who 
have  not  had  considerable  training 
or  who  are  not  highly  sophisticated 
instructional  designers. 

Military  instructor  personnel  have 
difficulty  using  schemes  of  this 
nature,  yet  they  do  the  majority  of 
instructional  and  test  development. 
In  addition,  the  implications  for 


^Portions  of  this  article  were  presented  at  the  American  Psychological 
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instruction  and  testing  are  often 
not  clearly  specified  (Briggs  1977). 
Merrill,  Richards,  Schmidt,  and 

Wood  (1977)  developed  a  classifica¬ 
tion  scheme  for  use  by  U.S.  Navy 
instructional  developers.  The  scheme 
was  tested  and  extensively  revised 
(Ellis  &  Wulfeck,  1979)  and  is 

currently  being  used  by  Navy  per¬ 
sonnel.  In  its  present  form,  it 

meets  the  two  conditions  listed 
above  (Ellis  &  Wulfeck,  1981). 

Classification  of  objectives  is 
necessary  for  several  reasons. 
First,  it  helps  make  more  precise 
judgments  about  the  accuracy  of 
learning  objectives  and  leads  to 

more  precise  test  item  specifica¬ 
tions.  It  also  helps  make  judg¬ 
ments  about  how  consistent  ob¬ 
jectives  and  test  items  are  with 
each  other.  If  we  didn't  classify 
objectives  and  test  items,  all  we 
could  say  is,  "This  is  an  objective 
and  this  is  a  test  item,  and  they 
don't  look  too  different.”  The 
classification  scheme  allows  us  to 
insure  that  objectives,  test  items, 
and  the  corresponding  instruction 
all  address  the  same  thing.  The 
classification  scheme  also  helps  us 
to  judge  whether  or  not  objectives 
and  test  items,  and  instructional 
presentations  are  adequate. 

The  Classification  Scheme 

Objectives  and  test  items  can  be 
classified  according  to:  (1)  What 
the  student  must  do;  that  is,  the 
task  to  be  performed,  and  (2)  the 
instructional  content;  that  is,  the 
type  of  information  the  student 
must  learn. 

The  Task  Dimension 

A  student  can  either  remember  infor¬ 
mation,  or  use  the  information  to 
do  something.  This  distinction 
corresponds  to  the  difference  be¬ 
tween  knowledge  and  application, 
and  to  the  difference  between 


declarative  and  procedural  know¬ 
ledge.  The  following  two  test 
items  illustrate  the  remeraber-use 
distinction. 

a.  Remember:  The  symbol  for  resis¬ 
tor  is _ . 

b.  Using  your  knowledge  of  elec¬ 
tronic  theory,  predict  what  would 
happen  in  the  circuit  shown  below 
if  the  load  resistance  were 
shorted? 

These  two  test  items  differ  with 
respect  to  what  the  student  is 
supposed  to  do  (Task).  In  the 
first  item,  the  student  has  to 
remember  something.  In  the 
second,  he  has  to  apply  or  use 
his  knowledge  in  a  new  situation. 

The  Content  Dimension 

There  are  five  types  of  content; 
facts,  categories,  procedures, 
rules,  and  principles.  Facts  are 
simple  associations  between  names, 
objectives,  symbols,  locations, 
etc.  Facts  can  only  be  remem¬ 
bered  while  the  other  content 
types  can  be  remembered  or  used. 
Categories  are  classifications 
defined  by  certain  specified 
characteristics.  Procedures  con¬ 
sist  of  ordered  sequences  of 
steps  or  operations  performed  on 
a  single  object  or  in  a  specific 
situation.  Rules  also  consist  of 
ordered  sequences  of  operations, 
but  can  be  performed  on  a  variety 
of  objects  or  in  a  variety  of 
situations.  Principles  involve 
explanations,  predictions  or 
diagnoses  based  on  theoretical  or 
cause-effect  relationships. 

Facts  can  only  be  remembered,  but 
for  the  other  content  types,  the 
student  may  be  asked  to  use  his 
knowledge  to  classify,  perform, 
solve,  or  predict. 

The  use 


level  can  be  further 


213 


divided  into  two  types:  (1)  Use-Un¬ 
aided,  where  the  student  has  no  aids 
except  his  own  memory,  and  (2)  Use- 
Aided,  where  the  student  has  a  job 
aid  to  perform  the  task.  For  the 
use-aided  type,  the  nature  of  the  aid 
depends  on  the  content  type.  For  use- 
aided  category,  the  aid  consists  of  a 
decision  strategy  including  each  crit¬ 
ical  characteristic.  In  simple  cases, 
when  the  aid  may  comprise  only  a  list 
of  characteristics,  the  decision 
strategy  would  be  implied.  For  use- 
aided  procedures,  the  aid  is  a  list 
of  steps  to  be  performed.  For  use- 
aided  rules,  the  aid  is  at  least  a 
statement  of  the  formula  or  rule  to 
be  applied ,  and  could  include  guide¬ 
lines  for  when  and  how  to  apply  it. 
For  use-aided  principles,  the  aid  is 
also  at  least  a  statement  of  the 
principle,  and  could  include  guide¬ 
lines  for  when  and  how  to  apply  it. 
In  summary,  the  remember  level  in¬ 
volves  "pure"  remembering;  the  use- 
unaided  level,  remembering  what  is  to 
be  used  and  then  using  it;  and  the 
use-aided  level,  “pure"  using. 

An  Overview  of  the  IQI  Procedures 

The  IQI  consists  of  five  procedures. 
Since  all  the  steps  of  the  instruc¬ 
tional  development  process  depend  on 
careful  specification  of  learning  ob¬ 
jectives,  the  first  procedure  is  to 
assure  the  adequacy  of  objectives. 
This  is  done  by  classifying  each  ob¬ 
jective,  determining  whether  all  the 
necessary  conditions,  standards,  and 
actions  are  present,  and  judging 
whether  or  not  the  objective  accu¬ 
rately  reflects  what  the  student  is 
supposed  to  do  or  know  following 
training. 

The  next  two  procedures  Involve  check¬ 
ing  test  items.  First,  test  item(s) 
are  evaluated  against  their  corres¬ 
ponding  objectives  to  ensure  that 
the  test  items  are  consistent  with 
the  objectives.  This  is  accomplished 
by  making  sure  that  the  conditions, 
standards,  and  actions  for  the  test 


items  are  the  same  as  those  for  the 
objective,  and  by  checking  to  see 
that  each  test  item  is  classified  in 
the  same  way  as  its  objective.  Next, 
the  adequacy  of  the  test  items  is 
assessed  by  determining  whether  each 
item  conforms  to  the  rules  for  prop¬ 
er  item  construction. 

The  final  two  procedures  are  con¬ 
cerned  with  the  instructional  pre¬ 
sentation.  The  term  "instructional 
presentation"  covers  all  the  ways 
instruction  can  be  presented.  This 
includes  printed  self-study  mater¬ 
ials,  lectures,  computer-assisted 
instruction,  films  or  videotapes, 
tape-slide  presentations,  audio 
tapes,  videodisc,  one-on-one  tutor¬ 
ials,  and  any  combination  of  these. 

At  this  point,  the  objectives  and 
test  items  for  the  instructional  pro¬ 
gram  are  consistent  with  each  other, 
and  are  adequate.  The  next  step  is 
to  make  sure  that  the  instructional 
presentation  is  consistent  with  the 
objectives  and  test  items.  For  a 
presentation  to  be  consistent  with 
an  objective/test  item,  it  must 
teach  to  the  task  level  and  content 
type  of  the  objective/ test  item. 
This  means  that  the  presentation 
must  contain  certain  components, 
depending  on  the  task/content  level. 
There  are  four  main  presentation  com¬ 
ponents:  statements,  examples,  prac¬ 
tice  remembering  and  practice  using. 
For  statements  the  student  is  given 
a  statement  of  fact,  a  category 
definition,  the  steps  of  a  procedure 
or  rule,  or  a  statement  of  a  prin¬ 
ciple.  For  examples  the  student  is 
told  or  shown  how  a  statement  of  a 
category,  procedure,  rule,  or  prin¬ 
ciple  applies  in  a  specific  case. 
For  practice  remembering  the  student 
is  asked  to  supply  part  or  all  of  a 
fact  statement,  category  definition, 
the  steps  of  a  procedure  or  rule,  or 
a  statement  of  a  principle.  The 
student  is  also  given  feedback  about 
the  correctness  of  his  answer.  For 
practice  using  the  student  is  asked 


to  use  a  category  definition,  pro¬ 
cedure,  rule,  or  principle  on  a 
specific  case  to  which  it  applies, 
and  is  given  feedback  about  the 
quality  of  his  performance.  Dif¬ 
ferent  combinations  of  these  com¬ 
ponents  are  required,  depending  on 
the  task  level  of  the  objective.  A 
component  is  counted  as  present  in 
the  instruction  only  if  it  is  com¬ 
plete,  This  depends  on  the  content 
type  of  the  objective,  and  simply 
means  that  everything  that  needs  to 
be  included  is  included. 

Instruction  can  be  consistent,  but 
still  not  teach  as  effectively  as 
it  could.  The  final  procedure  eval¬ 
uates  the  adequacy  of  the  instruc¬ 
tional  presentation.  The  adequacy 
procedure  incorporates  a  number  of 
instructional  design  principles 
which  have  been  shown  to  promote 
student  learning.  These  include 
guidelines  for  formatting  infor¬ 
mation  so  students  can  find  it,  for 
communicating  it  clearly  and  effec¬ 
tively  so  students  can  understand 


it,  and  for  including  additional 
explanation,  so  students  can  bet¬ 
ter  learn  and  remember  the  infor¬ 
mation. 

CONCLUDING  STATEMENT 

The  IQI  consists  of  four  volumes: 
I.  Introduction  and  Overview 
(Wulfeck,  Ellis,  Richards,  Wood, 
and  Merrill,  1978),  II.  Users 
Manual  (Ellis,  Wulfeck,  and 
Fredericks  1979),  III.  Training 
Workbook  (Fredericks,  1980),  and 
IV.  Job  Performance  Aid  (Ellis 
and  Wulfeck,  1978).  All  of  these 
documents  can  be  obtained  from 
the  Defense  Technical  Information 
Center,  Cameron  Station,  Alex¬ 
andria,  Virginia.  The  appropriate 
access  numbers  are  given  in  the 
reference  section.  Further  infor¬ 
mation  can  be  obtained  by  writing 
John  Ellis  or  Wallace  Wulfeck  at 
Code  13,  Navy  Personnel  Research 
and  Development  Center,  San  Diego, 
California  92152. 


REFERENCES 


Branson,  R.  K. ,  Rayner,  G.  T. ,  Cox,  J;  L.,  Furman,  J.P.,  King,  F.J. , 
and  Hannum,  W.  H.  Interservice  Procedures  for  Instructional  Systems 
Development:  Executive  Summary  Phase  I ,  Phase  II,  Phase  III,  Phase  IV, 
and  Phase  V.  Interservice  Training  Review  Organization  Report  (DTIC 
Nos.  AD  AO 19486  to  AD  A019490). 

Briggs,  L.  D.  Instructional  Design:  Principles  and  Applications, 
Englewood  Cliffs,  N.J.:  Educational  Technology  Publications,  1977. 

Ellis,  J.  A.,  Wulfeck,  W.  H.  and  Fredericks,  P.  The  Instructional 
Quality  Inventory:  II.  User's  Manual .  San  Diego,  CA:  Navy  Personnel 
Research  and  Development  Center,  Special  Report  79-24,  August  1979, 

(DTIC  No.  AD  A083678). 

Ellis,  J.  A.  and  Wulfeck,  W.  H.  The  Instructional  Quality  Inventory: 
IV.  Job  Performance  Aid.  San  Diego,  CA:  Navy  Personnel  Research  and 
Development  Center,  Special  Report  79-5,  November  1978,  (DTIC  No. 

AD  A083928) . 


215 


Ellis,  J.  A.  and  Wulfeck,  W.  H.  Quality  Control  of  Instruction  and 
Instructional  Design.  Paper  presented  at  the  American  Psychological 
Convention,  1981,  Los  Angeles,  CA. 

Fredericks,  P.  The  Instructional  Quality  Inventory:  III.  Training  Hand¬ 
book.  San  Diego,  CA:  Navy  Personnel  Research  and  Development  Center,  Special 
Report  80-25,  July  1980,  (DTIC  No.  AC  A092804). 

Gagne,  R.  M.  The  Content  Analysis  of  Subject  Hatter:  The  Computer  as  an 
Aid  in  the  Design  of  Criterion-Referenced  Tests.  Instructional  Science ,  1976, 

5 j  1-28. 

Merrill,  M.D. ,  Richards,  R.E. ,  Schmidt,  R.V.,  and  Wood,  N.D.  The 
Instructional  Strategy  Diagnostic  Profile  Training  Manual ,  San  Diego,  CA: 
Courseware,  Inc.,  1977. 

Wulfeck,  W.H. ,  Ellis,  J.A. ,  Richards,  R. ,  Wood,  N. ,  and  Merrill,  M.D. 

The  Instructional  Quality  Inventory :  I.  Introduction  and  Overview.  San 
Diego,  CA:  Navy  Personnel  Research  and  Development  Center,  Special  Report  79-3, 
November  1978,  (DTIC  No.  AD  A062493). 


PROSPECTS  FOR  ENERGY  FROM  COAL  SAID  TO  BE 
BETTER  THAN  FROM  NUCLEAR 

A  study  sponsored  jointly  by  the  U.S.  Department  of  Energy  and  the 
U.S.  Department  of  Labor  expects  a  decline  in  construction  of 
nuclear  plants  after  this  year  (1982),  with  construction  declining 
to  zero  by  the  mid-1990s.  By  the  year  1995  plants  using  coal  are 
expected  to  comprise  92  percent  of  power  plants  under  construction, 
me  study  was  conducted  by  Dr.  Roger  Bowl  by.  University  of 
Tennessee  Professor  of  Economics,  and  Drs.  Soon  Paik  and  William 
hcnriver,  U.S.  Department  of  Labor  economists.  A  summary  of  the 
report  was  included  in  the  TORCHBEARER,  a  publication  of  the 
University  of  Tennessee  at  Knoxville. 


